Programs as Black-Box Explanations

With increasing complexity of machine learning systems being used1, there is a crucial need for providing insights into what these models are doing. Model-agnostic approaches [18], such as Baehrens et al. [1] and Ribeiro et al. [17], have shown that insights into complex, black-box models do not have to come at a cost of accuracy, and that accurate local explanations can successfully be provided for a number of complex classifiers (such as random forests and deep neural networks) and domains (text and images) for which interpretable models have not performed competitively. However, we still need to identify which interpretable representation would be suitable to convey the local behavior of the model in an accurate and succinct manner, and existing model-agnostic approaches have focused only on (sparse) linear models. Work in interpretable machine learning, on the other hand, has proposed many more other representations when designing their models, ranging from additive models, to decision rules, trees, sets, and lists, amongst others [8, 10].